1. Technical Field
The present invention relates to an image processing method, particularly used in a vision-based localization of a device, which comprises providing at least one image of at least one object or part of the at least one object. The invention also relates to a computer program product comprising software code sections or hardware accelerated processing units for performing the method.
2. Background Information
Image processing including filtering and extraction of information from an image are key tools needed to solve higher level problems such as but not limited to the areas of computer vision, machine learning, robotics or general image enhancement. One of the numerous examples is the detection of edges in an image that can contain a known object or part of a known object, which can be used for matching these edges with edges of the known object to which a reference coordinate system is attached. Based on the matches between the edges in the image and the object itself the camera with which the image was captured can be localized with respect to the reference coordinate system of the known object.
Image processing, filtering and extraction of information from an image can be done in very different ways, such as but not limited to morphological operators and convolution based on kernels, and by applying or combining the same or different processing steps, filters or extraction methods on the intermediate results of an earlier or parallel processing step. Image processing, filtering and extraction of information can be done but is not limited to perform, analyze, extract or create intensities, gradients, derivatives, edges, lines, segments, corners, features, smoothing, blur, transformation, denoising, sharpening, erosion and/or dilation. Besides the isotropic image processing, filter or extraction methods some methods are anisotropic, such as but not limited to 1D or imageaxis-aligned convolution kernels, such as, e.g., gradient filter like the known Sobel, Prewitt or Scharr filter, which use image axis-aligned convolution kernels, which means that generally these kernels extract image-horizontal or image-vertical gradients of the image. Isotropic image processing, filter or extraction methods can also be adapted to be anisotropic, by for instance using a 1D convolution kernel instead of the 2D one, like for instance for a smoothing convolution kernel.
The following types of standard approaches exist for image processing in this context: In case the image processing operator of the image processing method comprises image edge detection, generally the image gradient computation comprises applying image filters, such as Sobel, Prewitt or Scharr filters, with kernels aligned with the image axes. This means that generally these kernels produce image-horizontal or image-vertical gradients (see upper row A in FIG. 3). These approaches do not consider any additional information, if any, on some degrees of freedom and, thus, by themselves do not detect any object properties with respect to a known orientation, such as gravity-aligned edges.
If based on these image-axis aligned kernel based filters edges of a specific orientation, such as the gravity aligned edges in the image, need to be filtered, additional post-processing of the earlier results of the image-axis aligned filters could be applied. For instance, the orientation of each edge point in the image can be computed based on the image-horizontal and the image-vertical gradient for that point. Finally the edge-orientation of each pixel in the image could be compared with the projected gravity orientation in the image. This means for this approach that the image-horizontal and image-vertical gradients would need to be computed beforehand by applying both axis-aligned kernels, and an additional post processing step of comparison would be needed to be applied. This would make such an approach computationally more expensive.
A second approach, which is using additional information on some degrees of freedom, is to first adapt the whole image to align with this additional information (see FIG. 4). For instance, in the case of constraining the image processing method to compute gravity aligned data, the whole image can first be rectified, such that one image-axis is aligned with the projected gravity vector in the image, and afterwards the image-axis aligned image processing can be applied. In the case of gradient filter this will produce gravity aligned gradients. This second approach has two drawbacks: First, the image filters are not processing on the original image anymore, and thus the results of the final filter depend on the intermediate results of the performed rectifying approach. Second, the rectification itself is needed and thus this approach is computationally more expensive than one directly processing on the original image.
It would therefore be beneficial to provide an image processing method which is capable of providing a more efficient result in terms of quality of its result and processing speed.